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Abstract 

In this paper, we study the Gauss-Newton method for a special class of systems of 
nonlinear equation. Under the hypothesis that the derivative of the function under con- 
sideration satisfies a majorant condition, semi-local convergence analysis is presented. 
In this analysis the conditions and proof of convergence are simplified by using a sim- 
ple majorant condition to define regions where the Gauss-Newton sequence is "well 
behaved". Moreover, special cases of the general theory are presented as applications. 

Keywords: Gauss-Newton method; majorant condition; nonlinear systems of equations; 
semi-local convergence. 

1 Introduction 

Consider the systems of nonlinear equations 

F{x) = 0, (1) 

where F -.Vt ^ is a continuously differentiable function and C is an open set. 

When F'{x) is invertible, the Newton method and its variant (see [5], |6l El |T0] ) are the most 
efficient methods known for solving ([T]). However, when F'{x) is not necessarily invertible, 
a generalized Newton method, called the Gauss-Newton method (see [HEIE]), defined by 

Xk+i = Xk- F'{xk)'^F{xk), A; = 0, 1,..., 
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where F'lxkY denotes the Moore- Penrose inverse of the hnear operator F'{xk), finds least 
squares solutions of ([1]) which may or may not be solutions of ([1]). These least squares 
solutions are related to the nonlinear least squares problem 

min (2) 

that is, they are stationary points of G{x) = It is worth noting that, if F'{x) is 

surjective, then least squares solutions of systems of nonlinear equations are also solutions 
of systems of nonlinear equations. 

We shall consider the same special class of systems of nonlinear equations studied in 
[TTl [T2I [H], i.e., systems of nonlinear equations where the function F under consideration 
satisfies 

\\F'{yyiI^^-F'{x)F'{xy)F{x)\\<K\\x~yl \/ x,y efi (3) 

for some < k < 1 and I^m denotes the identity operator on M"*. This special class of 
nonlinear systems of equation contains underdertermined systems with surjective derivatives, 
because when F'{x) is surjective we can prove that k = in 

In recent years, papers have addressed the issue of convergence of the Newton method, 
including the Gauss-Newton method, by relaxing the assumption of Lipschitz continuity of 
the derivative (see [2l[3lll[5l[6l[71[8l[9l[T0l[l2l[Tl[l6l[I71[ig references therein). These 
new assumptions also allow us to unify previously unrelated convergence results, namely 
results for analytical functions (a-theory or 7-theory) and the classical results for functions 
with Lipschitz derivative. The main new conditions that relax the condition of Lipschitz 
continuity of the derivative include the majorant condition, which we will use, and Wang's 
condition, introduced in p!6] and used for example in [El [T71 [18] to study the Gauss- Newton 
method. In fact, under the hypothesis in this paper, it can be shown that these conditions 
are equivalent. However, the formulation as a majorant condition is in a sense better than 
Wang's condition, as it provides a clear relationship between the majorant function and the 
nonlinear function under consideration. Besides, the majorant condition provides a simpler 
proof of convergence. 

Following the ideas of the semi- local convergence analysis in [SI , we will present a new 
semi-local convergence analysis of the Gauss- Newton method for solving ([T]), where F satisfies 
([2]), under a majorant condition. The convergence analysis presented here communicates the 
conditions and proof in a quite simple manner. This is possible thanks to our majorant 
condition and to a demonstration technique introduced in [TO] which, instead of looking only 
to the sequence generated, identifies regions where, for the problem under consideration, the 
Gauss-Newton sequence is well behaved, as compared with a method applied to an auxiliary 
function associated with the majorant function. Moreover, two unrelated previous results 
relating to the Gauss-Newton method are unified, namely, results for analytical functions 
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under an a-condition and the classical result for functions with Lipschitz derivative. Besides, 
convergence results for underdetermined systems with surjective derivatives will be also 
given. 

The paper is organized as follows. Sect. 11.11 lists some notations and basic results used 
in the presentation. Sect. [2] states and proves the main results. Finally, special cases of the 
general theory are presented as applications in Sect. [31 



1.1 Notation and auxiliary results 

The following notations and results are used throughout this presentation. Let M" be with 
a norm ||.||. The open and closed balls at a G M" and radius 6 > are denoted, respectively 

by 

5(a, 5) := {x e R"; \\x - a\\ < 5}, B[a, 6] := {x E M"; ||x - a\\ ^ 6}. 

Given a linear operator A : R" — > R™ (or an n x m matrix), the Moore- Penrose inverse 
of A is the linear operator A^ : R" (or an m x n matrix) which satisfies: 

AA^A = A, A^AA^ = A\ (AA^)* = AA\ {A^ A)* = A^A, 

where A* denotes the adjoint of A. The Kernel and image of A are denoted by Ker{A) and 
im{A), respectively. It is easily seen from the definition of the Moore- Penrose inverse that 

AA^ =UKeriA)^, A^A = Ui^^A), (4) 

where He denotes the projection of M" onto subspace E. 

We use I]Rm to denote the identity operator on M™. If A is surjective, then 

A^ = A*{AA*)-\ AA^ = Jm™, {AA^^ = AAK (5) 

Lemma 1. (Banach's Lemma) Let i? : — t- be a continuous linear operator. If \\B — 
< 1, then B is invertible and \\B~^\\ < 1/ (1 — — I^nW) . 

Proof. See the proof of Lemma 1, p. 189 of Smale |15j with A = I^n and c = — □ 

The next lemma is proved on p. 43 of p3] (see also [1]). It is on the perturbation of the 
Moore-Penrose inverse of A. 

Lemma 2. Let A, S : R" — )■ R™ be continuous linear operators. Assume that 

1 <rank{B) <rank{A), \\A^\\\\A - B\\ < 1. 

Then 

WBm 

rank{A) = rank{B), \\A^\ < — 



Bm\\A-B\ 
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2 Semi-local analysis for the Gauss-Newton method 

Our goal is to state and prove a semi-local theorem of the Gauss-Newton method for solving 
nonlinear systems of equations, where the function under consideration satisfies ([3]). First, 
we will prove that this theorem holds for an auxiliary function associated with the majorant 
function. Then, we will prove well-definedness of the Gauss- Newton method and convergence. 
Convergence rates will also be established. The statement of the theorem is: 

Theorem 3. Let Q C R" be an open set and F : — )• M™ a continuously dijferentiable 
function. Suppose that 

\\F'{yy{h^-F'{x)F'{xy)F{x)\\<K\\x-yl Vx,y en (6) 

for some < k < 1. Take xq E Q such that (3 := ||F'(a;o)^-F(a;o) || > 0, F\xq) ^ and 

rank{F'{x)) <rank{F'{xo)), ^ x e Q. (7) 

Suppose that there exist R > and a continuously differentiahle function / : [0, i?) — > M 
such that, B{xo,R) C 

||F'(xo)1 WF'iy) - F'ix)\\ < f'iWy - a;|| + ||x - Xo\\) - - Xo\\), (8) 

for any x,y E B{xo, R), \\x — Xo\\ + \\y — x\\ < R, 

hi) /(O) = 0, /'(O) = -1; 

h2) /' is convex and strictly increasing. 

Take A > such that A > —K,f'{f3) and consider the auxiliary function hp^x : [0, i?) — M, 

VaW :=/3 + At + /(t). (9) 

If hii,\ satisfies 

h3) hp^xit) = for some t G (0, R), 

then hi3^x{t) has a smallest zero t* G {0,R), the sequences for solving hp^xit) = and 
F{x) = 0, with starting point to = and xq, respectively, 

tk+i = tk-h'p Q{tkY^hii^x{tk), Xk+i = Xk- F'{xk)^F{xk), A; = 0, 1, . . . , (10) 
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are well defined, {tk} is strictly increasing, is contained in [0,t*), and converges to t*, {xk} 
is contained in B{xQ,t^,), converges to a point G B[xo,t^,] such that F'lx^^Y F{x^) = and 

\\xk+i — Xk\\ < tfc+i — tk, — Xk\\ < t^ — tk, k = 0,1, . . . , (11) 

\\xk+i — Xk\\ < i,^, T^W^k — Xk-i\\'^ , k = 1,2, . . . . (12) 

[tk ~ L-k-l) 

Moreover, if X = (X = and h'^ ^it^) < 0), the sequences {t^} and {xk} converge Q-linearly 
and R-linearly (Q-quadratically and R-quadratically) to t^ and x^,, respectively. 

Remark 1. It is easily seen that the best choice of X is the smallest possible. Hence, if 
f'{l3) < then X = -Kf'{l3) is the best choice. Moreover, smce -fi/S) < -/'(O) = 1 (h2), 
a possible choice for X is k, despite not being the best. 

Remark 2. If F\x) is surjective, it follows from the second equation in (|5]) that F' [x)F\x)^ = 
/]Rm. Thus, we can take A = 0, because F satisfies with k = 0. Therefore, in this case, 
Theorem\^ extends the results obtained by Ferreira and Svaiter in Theorem 2 of [7^ . 

From now on, we assume that the hypotheses of Theorem [3] hold. 
2.1 The auxiliary function and sequence {tk\ 

In this section, we will study the auxiliary function, hp^\, which is associated with the 
majorant function, /, and prove all results regarding only the sequence {tk}- Remember 
that a function that satisfies ([8]), hi and h2 is called a majorant function for the function F 
on B{xq, R). More details about the majorant condition can be found in [H [5l [6], El [HI [9l [TO]. 

Proposition 4. The following statements hold: 

i) /i^,a(0) = /3 > 0, /i^^,(0) = A - 1; 

ii) h'p )^ is convex and strictly increasing. 

Proof. It follows from the definition in ([9]) and assumptions hi and h2. □ 

Proposition 5. The function hj^^x has a smallest root t^, G {0,R), is strictly convex, and 

hp^xit) > 0, h'^^oit) < 0, t<t- hp^x{t)/h'p^o{t) <t„ V t G [0, t,). (13) 

Moreover, h'^ ^i^t^) < 0. 
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Proof. As hp^x is continuous in [0,R) and have a zero there (h3), it must have a smallest 
zero t*, which is greater than because /i^,A(0) = (3 > 0. Since h'^^ ^ is strictly increasing by 
item ii of Proposition |H /i^ ^ is strictly convex. 

The first inequality in ( |T3l) follows from the assumption /i/3,a(0) = /3 > and the definition 
of as the smallest root of /i^^a- Since hjs^x is strictly convex, 

= VA(i*) > VaW + ^;3,aW(^*-^), te[0,R),t^U. (14) 

If t G [0,t*) then /i/3,A(i) > and - t > 0, which, combined with ([H]) yields h'p^^{t) < 
for all t G [0,t*). Hence, using A > and h'^y^if) = A + h'^^it) for all t G [0,t*) the second 
inequality in (fT3|) follows. The third inequality in (fT3|) follows from the first and the second 
inequalities. 

To prove the last inequality in ( fT3l) . note that the division of the inequality on (fT4l) 
by — /i^ a(^) (which is strictly positive), together with some simple algebraic manipulations, 
gives 

t- VaWA^aW <^*' VtG[0,t,), 

which, using the first inequality in (fT3|) and < —h'f^ y^it) < —h'^fiit) for all t G [0,t*), yields 
the desired inequality. 

Since hjs^x > in [0,t*) and /i/3,a(^*) = 0, we must have h'f^^^it^) < 0. Thus, the last 
inequality of the proposition follows from the fact that h'f^ y^it^) = A + h'^ ^it^). □ 

In view of the second inequality in (fT3|) . the following iteration map for h^^x is well defined 
in [0,t*). Denoting this by ^: 

rih,^, : [0,Q ^ R 

t ^ t-h^,x{t)/h'^^,{t). ^'^^ 

Note that in the case where A = 0, the sequence n/t^ ^ reduces to a Newton sequence, which 
Ferreira and Svaiter used in [10] to obtain a semi-local convergence analysis of the Newton 
method under a majorant condition. 

Proposition 6. For each t G [0,t*) it holds that (3 < n^i^ ^{t) < t^. 

Proof. Proposition [5] implies that hjs^x is convex. Hence, using item i of Proposition H] it is 
easy to see, by using convexity properties, that (1 — \)t — /3 > —hp^xit)^ which combined 
with A > gives t — (3 > —hj3^x(t)- Accordingly, the above definition implies that 



n.,,{t) -P = t- ^ - P > -h,At) - ^ = ^r^[^:^.o(t) + 1], V t G [0, t.). 
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Proposition m implies that /i^o(O) ~ ^"^^ ^/3o strictly increasing. Thus, we obtain 
^/3o(^) + 1^0) foi' 9-11 t ^ [0,t*). Therefore, combining the above inequality with the first 
two inequalities in Proposition [5l the first inequality of proposition follows. To prove the 
last inequality of proposition, combine ( IT5l) with the last inequality in ( 1131) . □ 

Proposition 7. Iteration map maps [0,t*) in [0,t*), and it holds that 

tKTih^Jt), VtG[0,t,). 
Moreover, if X = or X = and /i^o(^*) ^ '^^ have the follows inequalities, respectively. 



t*-nH,,,{t) ^ liU-t), t,-nh,Jt) < £-^h^^t,_tf, VtG [0,t, 



Proof. The first two statements of the proposition follow trivially for the last inequalities 
in ffT^ and ffT^ . Now, if A = 0, then the sequence in f[T^ reduces to a Newton sequence. 
Hence, the second part of the proof follows the same pattern as the proof of Proposition 4 
of [lO] with hpfl = f. □ 

The definition of {tj.} in Theorem [3] is equivalent to the following one 

to = 0, tk+i = nhp^^{tk), fc = 0,l,.... (16) 
Therefore, using also Proposition [7] it is easy to prove that 

Corollary 8. The sequence {tk} is well defined, is strictly increasing, is contained in [0,t*), 
and converges to t^,. 

Moreover, if X = or X = and /i^o(^*) ^ ^> sequence {tk} converges Q-linearly or 
Q- quadratically to t^, respectively, as follows 

1 D'h'p^^it^) 2 

t* — tk+l < —{t* ~ tk), t* — tk+l ^ „, , ' / N (t* ~ tk) , k = 0, 1, . . . . 

z —ZripQit^) 
Hence, all statements involving only {tk} on Theorem [3] are valid 



2.2 Convergence 

In this section we will prove well definedness and convergence of the sequence {xk} specified 
on flTU]) in Theorem [31 

We start with two lemma that highlight the relationships between the majorant function 
/ and the non-linear function F. 
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Proposition 9. // ||a; — xo|| < t < t^, then rank{F'{x)) = rank{F'{xo)) > 1 and 

\\F\xy\\<-\\F\xoy\\/h',^o{t)- 

In particular, rank{F'{x)) = rank{F'{xo)) in B{xQ,t^). 

Proof. Take x G B[xo,t], <t <t^. Using the assumptions ([8]), hi, h2, f'{t) = h'^fiit) and 
the second inequality in f|T3|) we obtain 

||F'(xo)^|| WF'ix) - F'ixo)\\ ^ f'iWx - xoW) - /'(O) ^ fit) + 1 = h'^^.it) + 1< 1. 

Combining the last inequality with (I7j) and Lemma [2], we conclude that rank{F'{x)) = 
rank{F' (xo)) > 1 and 

.t|| < ll-^X3:o)t|| \\F'ixo)m ||FXxo)t|| 

" ^ ^ " ^ 1 - Wit) + 1) -fit) h'^^,{t) ■ 

□ 

It is convenient to study the linearization error of F at point in Vt. For that purpose we 
define 

Ef{x, y) := F{y) - [F{x) + F\x){y - x)] , y,xe il. (17) 
We will bound this error by the error in the linearization on the majorant function / 

efit, u) := f{u) - [/(t) + f\t){u - t)] , t,ue [0, R). (18) 

Lemma 10. Take 

x,y & B{xo, R) and < t < v < R. 
If \\x — XqW and \\y — x\\ ^ v — t, then 

12 



\F'{xomE^{x,y)\\^ej{t, 



\y - x\ 



(v - ty 

Proof. The proof follows the same pattern as the proof of Lemma 7 of [10] ■ □ 



Proposition |9] guarantees, in particular, that rank{F\x)) > 1 for all x G B{xo,t^) and, 
consequently, the Gauss-Newton iteration map is well-defined. Let us call Gp, the Gauss- 
Newton iteration map for F in that region: 

GF:B{xo,t,) ^ 

X ^ x-F'{xyF{x). ^ ' 



One can apply a single Gauss-Newton iteration on any x G B{xo,t^) to obtain Gf{x) which 
may not belong to i?(xo,t*), or even may not belong to the domain of F. Therefore, this is 
enough to guarantee well definedness of only one iteration. To ensure that Gauss-Newton 
iterations may be repeated indefinitely, we need the following result. 

First, we define some subsets of B{xo, t*) in which, as we shall prove, the desired inclusion 
holds for all points in these subsets. 

Kit):=l^xen : ||x-Xo|| <t, ||F'(a;)tF(x)K-^f^|, t e [0,Q , (20) 
K:= [j K{t). (21) 

In ([20]), ^ t < t*, therefore, /i^o(t) ^ and rank{F\x)) > 1 in B[xQ,t] C B[xo,U) 
(Proposition [9]). Hence, the definitions are consistent. 

Lemma 11. For each t G [0,t*), it holds that: 

i) K{t) C B{xM; 

ii) ||GHGHx))-GHx)||<-^5f^ v.Gi^(t), 

ill) G^(ir(t))cir(n,,,(t)). 
As a consequence, K C B{xo,t^) and Gf{K) C K. 



Proof. Item i follows trivially from the definition of K{t). 

Take t G [0,t*), x G K{t). Using definition (l20l) and the first two statements in Proposi- 
tion [7] we have 

\\x - xoll < t, \\F'{x)^F{x)\\ < -VA(t)//i^,o(^), t < nk,,,{t) < t,. (22) 

Therefore 

||Gi?(a;) — Xoll ^ \\x — xo\\ + \\Gf{x) — x\\ = \\x — xq\\ + ||F'(a;)^F(a;) || 
^ t - hp,x{t)/h'^^^{t) = rih^ ^it) < t, , 

and 

Gf{x) G 5[xo,n;,^,,(t)] C 5(xo,t*). (23) 
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Since Gf{x), rih^^ ^{t) belong to the domains of F and /, respectively, using the definitions in 
f|T5|l and f|T9l) . hp^xif) = + At + /(t), linearization errors f|T7|) and f|T8|) and some algebraic 
manipulation, we obtain 

hpArihp^M) = hp^xiuhp^^it)) - [hp^x(t) + /i^,o(^)K;9.A W - ^)] 

= e/(t,n,^,,(t)) - Xhp,x{t)/h'p^^{t) (24) 

and 

F(G^(x)) = F{Gf{x)) - [F{x) + - x)] + (Jm., - F\x)F\xY)F{x) 

= Ef{x, Gf{x)) + ih^ - F'{x)F'{x)^)F{x). 

The last equation, together with simple algebraic manipulations, implies that 

\\F'{GAx)yF{GAxm < \\F\GF{x)y\\\\EAx,GF{xm 

+ ||F(G^(x))t(/K,„ - F'(a;)F'(x)t)F(x)||. 

As [[^^(a;) — a;o|| < Uhi^^it), it follows from Proposition [9] that rank{F' {G f{x))) > 1 and 

||F(G^(x))t||<-||F(a:o)lA;,,oK,,(t)). 

From the two latter equations and we have 

\\F\GF{x)yF{GF{x))\\ < - J^^^M^||E(x,G^(x))|| 

On the other hand, using (j22l) . Lemma fTOl and (!24ll we have 



||F'(xo)l||i?i.(x,G^(x))|| <e;(t,n,,Jt)) 



Thus, the last two equations, together with the second equation in ([22]), imply 



\\F'{GF{x)VF{GF{xm< — 



+ + A/i;3,oK,A W)~')(-VAW//^koW)- 
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Taking A > —Kf'{f3), the second inequality in f lT3|) and fl22|) . we obtain 

As f'{t) = h'pQit), using Proposition [6l h2 and the second inequahty in f|T3|) . we have 

k(i - f (/3)(/^;j,oK,,(t))-') = - /i;3,oK,.W))(-^koK,.W)"' < o- 

Combining the three above inequahties we conclude 

F [Gf{x)YF[Gf{x))\\ < -r— 7-— 7- . 

Therefore, item ii follows from the last inequality and f lTI?]) . Now, the last inequality combined 
with f lT^ . f[T^ and the second inequality in f l^ becomes 

This result, together with f l23|) . shows that (^^^(a;) G K{nhp ^{t)), which proves item iii. 

The next inclusion (first on the second part), follows trivially from definitions f l20|) and 
f l2T|) . To check the last inclusion, take x & K. Then x G -ft'(t) for some t G [0,t^,). Using 
item iii of the lemma, we conclude that Gf{x) G K{nh^ ^{t)). To end the proof, note that 
Ufi^ ^{t) G [0,t*) and use the definition of K. □ 

Finally, we are ready to prove the main result of this section, which is an immediate 
consequence of the latter result. First note that the sequence {x^} (see ffTOl) ) satisfies 

Xk+i = GF{xk), A; = 0,1,..., (25) 

which is indeed an equivalent definition of this sequence. 

Corollary 12. The sequence {xk} is well defined, is contained in B{xo,t^), converges to a 
point a;* G i?[xo,t*] such that F'{x^,y F{x*) = 0, and {xk} and {tk} satisfy ffTTl) and f|T2l) . 

Moreover, if \ = ( \ = and /i^^ol'^*) < Oy'? sequences {tk} and {xk} converge 
Q-linearly and R-linearly (Q-quadratically and R-quadratically) to t^, and x^, respectively. 

Proof. Since ||F'(xo)^-F(a;o)|| = /3, using the item i of the Proposition HI we have 

xo G K{Q) C K, 
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where the second inclusion follows trivially from f l2T|) . Using the above equation, the inclu- 
sions Gf{K) C K (Lemma [TT|) and fl25|) . we conclude that the sequence {xk} is well defined 
and lies in K. From the first inclusion in the second part of Lemma [TTl we have trivially 
that {xk} is contained in 5(xo,t*). 
We will prove, by induction, that 

XkeKitk), A; = 0,1,.... (26) 

The above inclusion, for A; = 0, is the first result in this proof. Assume now that Xk G K{tk). 
Thus, using item iii of Lemma [TT| f|T6|) and (l25l) . we conclude that Xk+i € K{tk+i)„ which 
completes the induction proof of fl26l) . 
Now, using ( l26|) and ( l20l) . we have 

||F'(xfc)^F(x,.)|| < -VA(4)//i;3,o(4), A; = 0, 1, ... , 
which, using (ITUl) . becomes 

- Xfcll < tfc+i - tfc, fc = 0, 1,.... (27) 
So, the first inequality in f lTT]) holds. As {t^} converges to t*, the last inequality implies that 



oo 



- < ^ t^+i -tk = U- tk,, < +oo, 

k=ko k=ko 

for any A;o G N. Hence, {xk} is a Cauchy sequence in B{xo,t^), and so converges to some 
x^: G i?[xo,t*]. The last inequality also implies that the second inequality in fllip holds. 

To prove that F'{x^yF{x^) = 0, note that, with simple algebraic manipulation, (jS]) and 
f lTOj) . we obtain 

\\F'{x,yF{xk)\\ ^ \\F'{x.y{l~F'{xk)F'{xky)F{xk)\\ 
+ \\F'{x,y\\\\F'{xk)F'{xkyF{xk)\\ 
< K,\\xk - + ||F'(x*)^||||F'(xfc)||||xfc+i - Xk\\. 

Due the fact that F is continuously differentiable, we can take limit in the last inequality to 
conclude that F'{x^y F{x^) = 0. 

Since Xk G K{tk), for all k = 0, 1, . . . , the inequality in f lT2|) . follows by applying item ii 
of the Lemma [TT] with x = Xk-i and t = tk-i and using the definitions in f|T6l) and fl25l) . 

To end the proof, combined the second inequality in flTT]) with the last part of the Corol- 
lary [SI □ 

Therefore, it follows from Corollaries |8] and [12] that all statements in Theorem [3] are valid. 
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3 Special cases 

In this section, we present some special cases of Theorem [3l 
3.1 Convergence result for F'{xo) surjective 

In this section we present a theorem under the hypothesis that F'{xq) is surjective. In this 
case, we can use a majorant condition, which gives the propriety that {xk} is invariant under 
the function F — >■ A'^F, where A : M" — )■ is any surjective linear operator. 

Theorem 13. Let Q C R" be an open set and F : Q ^ R"* a continuously differentiable 
function. Take xq & Q such that (3 := ||F'(xo)^-F(a;o) || > and F\xq) is surjective. Suppose 
that there exist R > and a continuously differentiable function / : [0, i?) — )■ M such that, 
B{xo,R) c n, 

\\F\xo)\F'iy) - F\xm < /'(lb - x|| + ||x - Xo||) - - Xo||), (28) 
for any x,y E B{xo, R), \\x — Xo|| + \\y — x\\ < R, 
hi) /(O) = 0, /'(O) = -1; 
h2) /' is convex and strictly increasing. 
Consider the auxiliary function hp : [0, R) — t- M, 

hit) -.= 13 + fit). (29) 

// hp satisfies 

h3) hpit) = for some t G (0, R), 

then hpit) has a smallest zero U G (0, R), the sequences for solving hpit) = and F(x) = 0, 
with starting point to = and xq, respectively, 

tk+i = tk - h'pitk)'^hpitk), Xk+i = Xk - F'{xk)^F{xk), k = 0,l,..., (30) 

are well defined, {tk} is strictly increasing, is contained in [0,t*), and converges Q-linearly 
to U, {xk} is contained in i?(xo,t*), and converges R-linearly to a point G B[xo,U] such 
that F'{x^)^F{x^) = 0, 

ll^fc+i ~ 3^fc|| ^ ^fe+i ~ ^fe) ll^;* ~ 3:^11 < t * — tk, k = 0, 1, . . . , (31) 
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I II ^ ^fe II l|2 ; _ 1 o 

\^k+\ — ^k\\ ^ 77 7 7?IFA: ~" ^k~\\\ , ft — i, /, 



(^fc — tk-l, 



\F\x,)^F{x,)\\ < ( J^ti^ ) ||F'(a;o)^F(a;,_i)||, A; = 1,2,... . (32) 



th — t 



k-l 



If, additionally, h'^it^) < 0, then the sequences {tk} and {xk} converge Q-quadratically and 
R-quadratically to and x^, respectively. 

Proof. Let F : ^ M'" be defined by 

F{x) = F'(xo)^F(x), xen. (33) 

Under the hypothesis of the theorem, we will prove that F satisfies all assumptions of the 
Theorem [3l Hence, with the exception of fl32|) . the statements of the theorem follow from 
Theorem [31 

First of all, as F'{xq) is surjective, it follows from that 

F\xo)F'{xoy = Jr™, {F'{xo)F'{xoyy = F\xo)F'{xoy. (34) 

Now, take x G B[xo,t], < t < t*. Using the assumptions fl25]) . hi and h2, we obtain 

WF'ixoYlF'ix) - F'(xo)]|| ^ f'iWx - xoll) - /'(O) ^ f'{t) + 1< 1. 

Using Lemma dl the above equation and the first equation in (IBljl . we conclude that (Jign — 
F' (xoY (F' (xq) — F'(x))) is non-singular and 

\\{lun-F'{xo)HF'{xo)-F'{x)))-'\\ ^ ^ _ ^ = (35) 

Again, the first equation in implies that F'{x) = F'(xo)(/Rn — F' {xoY {F' (xq) — F'{x))), 
which, using F'{xo) is surjective and (Jku — F'(xo)^(-F'(xo) — F'{x))) is non-singular, yields 
F'{x) is surjective for all x G B{xo, t*). Hence, using f p3|) and properties of the Moore-Penrose 
inverse, we have 

{F'{x)y = {F'{xoyF'{x)y = F'{xyF'{xo), Wxen. 

The latter inequality implies that F' satisfies ([6]) with k, = and the second sequence in fl30|l 
coincides with the second sequence in f lTO|) . Moreover, using fl33|) . and ([1]), we obtain 

\\F'{xoyF'{xo)\\ = UF'{xoyF'{xo)yF'{xoyF'{xoy\ = \\F'{xoyF'{xo)\\ (36) 



14 



and 

||F'(a;o)1 = \\F'{xoyF'{xo)\\ = l|n^,,(^,(,„))x || = 1. (37) 

Accordingly, §^ implies that ||F'(xo)_^F'(xo) || > 0, and §7^ together with ([28]) and ([33]) 
implies that F' satisfies ([H]) with / = /. 

Therefore, with the exception (1321) . the result of the theorem follow from Theorem [3] with 
F = F, f = f, h/B^x = h(s, X = and U = U- 

Our task is now to show that fl32]) holds. 

Take k G {1,2,...}. Using the first equation in flM]) . it follows by simple calculus that 
F'(xfc_i)tF'(xo)(/Mn - F'(xo)^(F'(xo) - F'(xfc„i))) = F' (xk-i)^ F' (xk-i), 
which, combined with ([!]), (1351) and ||a;fc-i — xo|| < tk-i < t*, yields 

||F'(xfc_i)tF'(xo)|| < ||n^,,(^,(,^_^))x(/Mn -F'(xo)^(F'(xo) -F'(xfc_i)))~'|| 
< - F'ixoYiF'ixo) - F'{x,^,))y'\\ 

Hence, using flHU]) and the first equation in flMl) . we obtain 

Ikfc = ||F'(xfc_i)tF(xfc_i)|| < -(/i;,(4_i))-iF'(xo)^F(x,_i)l|. (38) 

Since F{xk-i) is also surjective, it follows from ([5]) that F'{xk-i)F'{xk-iy = /r™, which 
combined with Lemma [10] and fl3Tl) gives 

\\F\xoyF{xk)\\ = \\F'{xoy{F{xk) - F{xk-i) - F'{xk-i){xk - Xk-i)\\ 
= \\F'{xoy\\\\EF{xk-i,Xk)\\ 

\Xk ■^/c— 1 II 



(tk — tk-i] 

Xk Xk—1 



(tk — tk-l) 

where the latter equation is obtained by combining (|T8l) . ( I29l) and (l30l) . Taking into account 
the last inequality, ( l38l) . {t^} and /i^ are strictly increasing, we have 



< 



h'p{tk-i) {tk — tfc-i) 
/i;3(tfc) ||F'(xo)tF(xfc_i) 



h'p{tk) {tk — tk-i) 

Therefore, the last inequality, together with the definition of {tk} in (l30l) . imply the desired 
inequality. □ 
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3.2 Convergence result for Lipschitz condition 

In this section, we first present a theorem corresponding to Theorem [3l but under the 
Lipschitz condition instead of the general assumption (jH]). We also present a theorem corre- 
sponding to Theorem ( fT3i) . but under the Lipschitz condition instead of assumption ( l28l) . 

Theorem 14. Let Q C M" be an open set and F : — > a continuously differentiable 
function. Suppose that 

\\F'{yy{h^-F'{x)F'{xy)F{x)\\<K\\x-yl Vx,y en 

for some < k < 1. Take xq E Q such that (3 := ||F'(a;o)^-F(a;o) || > 0, F'{xq) ^ and 

rank{F'{x)) < rank{F' {xq)) , V a; G fi. 

Suppose that there exist R> and L > 0, such that B{xo, R) C 

\\F'{xoy\\\\F'{x)-F'{y)\\ < L\\x-y\\, \/ x,y e B{xo,R) 

Take A = (1 — I3L)k, and consider the auxiliary function h/s^x : [0, R) — > R, 

hpAt) :=/3-(l-A)t + (Lt2)/2. 

// 



(/t2 - K + 1) + V2k2 -2k + 1' 



then hi3^\{t) has a smallest zero = (1 - A - - A)2 - 2/31) /L, the sequences for solving 
hp,\{t) = and F{x) = 0, with starting point to = and Xq, respectively, 

tk+i = tk- h'i^Q{tk)~^hp^x{tk), Xk+i = Xk - F'{xkyF{xk), k = 0,l,..., 

are well defined, {tk} is strictly increasing, is contained in [0,t*), and converges to t^, {xk} 
is contained in B{xQ,t^), converges to a point G B[xQ,t^] such that F'{x^y F{x^) = and 

\\Xk+l Xk\\ ^ tk+l tk, Xfcll ^ tk, k = 0, 1, . . . , 

11 II , '^fc+l ~ ^A: II II 2 7 1 o 

\\Xk+l - Xk\\ < — - Xk-l\\ , k = 1,2,... . 

[tk — tk-l) 

Moreover, if \ = (X = and h'^ ^it^) < 0^, then the sequences {tk} and {xk} converge 
Q-linearly and R-linearly (Q-quadratically and R-quadratically) to t^ and x^,, respectively. 
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Proof. It is immediate to prove that F, xq and / : [0, i?) — > M defined by f{t) = Lt'^/2 — t, 
satisfy tfie inequality ([8]), conditions hi and h2. Hence, 

VaW := /3 - (1 - A)t + (Lt2)/2 = (3 + Xt + fit). 

Since, 

/3L<A = ^ / = ^ / < 1, 39 

(k^ -K + l) + V2k2 -2k+1 (1 - k)2 + k + V2/t2 - 2k + 1 ~ 

we liave A = (1 — > and A = —K,f'{f3). Moreover, tlie first inequality in (139|) implies 
that (1 - - 2I3L > 0, i.e., h,s,\ satisfies h3 and = (l - A - ^^(1 - A)^ - 2/3L) /L is its 
smallest root. 

Therefore, taking /, /i/j^a, A and as defined above, all the statements of the theorem 
follow from Theorem [3l □ 

Under the Lipschitz condition. Theorem [T^ becomes: 

Theorem 15. Let Q C R" be an open set and F : Q ^ R™ a continuously differentiable 
function. Take Xq G Q such that (3 := ||F'(a;o)^-F(xo)|| > and F\xq) is surjective. Suppose 
that there exist R> and L > 0, such that B{xo, R) C f2, 

||F'(xo)^(F'(x) - F'{y))\\ < L\\x - y\\, ^ x,y e B{xo, R) 

Consider the auxiliary function hp : [0, R) — )■ R, 

hp{t) := f3-t+{Lt^)/2. 

If /3L < 1/2, then h 13 (t) has a smallest zero t:^ = (l — v^l — 2f3L)/L, the sequences for solving 
hp{t) = and F{x) = 0, with starting point to = and Xq, respectively, 

tk+i = tk-h'p{tkY^hp{tk), Xk+i = Xk- F'{xk)^F{xk), /i; = 0, 1,..., 

are well defined, {tk} is strictly increasing, is contained in [0,t*), and converges Q-linearly 
to U, {xk} is contained in B{xo,U), and converges R-linearly to a point x* G B[xo,U] such 
that F'{x^yF{x^) = 0, 

\\xk+i — Xk\\ ^ ife+i ~ tk, \\x^ — Xk\\ ^ t^ — tk, k = 0, 1, . . . , 

11 II ^ tk+l — tk ||2 , , „ 

\\Xk+i - Xk\\ < 77 r^\\Xk - Xk-i\\ , fc = 1,2, . . . , 

[tk — tk-l) 

\\F\xoyF{xk)\\< d'^'-^' ] \\F'{xoyF{xk-i)\\, k = l,2,.... 

\tk — tk-1 J 

If, additionally, PL < 1/2,, then the sequences {tk} and {xk} converge Q-quadratically and 
R-quadratically to and x^, respectively. 

Proof. The proof follows the same pattern as the proof of the Theorem [TH □ 
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3.3 Convergence result under Smale's condition 

In this section, we first present a theorem corresponding to Theorem [3|, but under Smale's 
a-condition, see |2l |3l US]. We also present a theorem corresponding to Theorem ( fT3l) . but 
under Smale's a-condition instead of the assumption ( l28i) . 

To simplify, we take A = k in the next theorem. As seen in Remark [H this is always a 
possible choice for A. 

Theorem 16. Let Q C MJ^ be an open set and F : Q ~¥ R™' an analytic function. Suppose 
that 

\\F'{yy{h^-F'{x)F'{x)^)F{x)\\<K\\x-y\\, \/ x,y Gfi 
for some < k < 1. Take Xq E Q such that (3 := ||F'(a;o)^-F(a;o) II > 0, F\xq) ^ and 

rank{F'{x)) < rank{F' (xq)) , V a; G fi. 

Suppose that 

^ < +00, B{xo, 1/7) C Q. (40) 



7 := ||F'(xo)1 sup 

n>l n\ 

Consider the auxiliary function hf^^K '■ [0, I/7) M, 

■.= P-{2-^i)t + t/{l-jt). 

If 



a 



:= /97 < 3 - 2V2, 



then hi3^i^{t) has a smallest zero t^, = (l — k + a — A/(r~^-"K+~ap"-^^(2^-"K)a) / (27(2 — k)), 
the sequences for solving hj^^^^it) = and F{x) = 0, with starting point to = and Xq, 
respectively, 

tk+i = tk-h'pQ{tkY^hii^^{tk), Xk+i = Xk- F'{xk)^F{xk), A; = 0,1,..., 

are well defined, {tk} is strictly increasing, is contained in [0,t*), and converges to t^, {x^} 
is contained in B[xQ^t^), converges to a point G i?[xo,t*] such that = and 

\\Xk+l 3^fc|| ^ ^fe+l tk, ||x* Xfcll ^ tk, /c = 0, 1, . . . , 

II II ^ — tk II ||2 u — ^ o 

\\Xk+l ~ Xk\\ ^ 77 7 II^^A: ~ II , K— i, /,.... 

Moreover, if k, = (k, = and /i^o('^*) < OA then the sequences {tk} and {xk} converge 
Q-linearly and R-linearly (Q-quadratically and R-quadratically) to t^ and x^,, respectively. 
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We need the following results to prove the above theorem. 

Lemma 17. Let C M" 6e an open set and F : Q ^ M"* an analytic function. Suppose 
that xq G M" and 7 is defined in ( 1411) . Then, for all x G B{xq, I/7) it holds that 

\\F'{xoy\\\\F"{x)\\<:{2j)/{l-j\\x-xo\\f. 
Proof. The proof follows the same pattern as the proof of Lemma 21 of [9]. □ 

Lemma 18. Let Q C M" be an open set and F : Q ^ M*" be twice continuously difjerentiable 
on Q. If there exists a / : [0, i?) — )■ M twice continuously difjerentiable and satisfying 

\\F\xoy\\\\F"{x)\\^f"{\\x-xo\\), 

for all X & Q such that ||x — Xo|| < R, then F and f satisfy ([H])- 

Proof. The proof follows the same pattern as the proof of Lemma 22 of [9] . □ 
Proof of Theorem 1161 Consider the real function / : [0, 1/7) — )■ M defined by 

It is straightforward to show that / is analytic and that 

/(0) = 0, f(t) = l/(l-7t)^-2, /'(0) = -l, r(t) = (27)/(l-7tf, nO) = n\r-\ 

for n > 2. It follows from the latter equalities that f satisfies hi and h2. Moreover, as 
f"{t) = (27)/(l — 7^)^, combining Lemmas [T71 and [T8l we have F and / satisfy ([8]) with 
R = 1/7. Hence, 

hp,x{t) -.= 13 -{2- X)t + t/{l - 7t) = /3 + At + f{t). 

Since A = k, we have < A < 1 and A = — k/'(0) > —Kf'{l3), where the latter inequality 
follows from h2. Moreover, a = /37 < 3 — 2^/2 implies that ((1 — k + a)^ — 4(2 — K)a) > 0, 
i.e., hi3^x satisfies h3 and t^, = (l — k + a — a/(1 — k + a)^ — 4(2 — K)a) / (27(2 — k)) is its 
smallest root. 

Therefore, taking /, h^^x, A and as defined above, all the statements of the theorem 
follow from Theorem 121 □ 
Under the Smale's a-condition. Theorem [T3] becomes: 
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Theorem 19. Let il C be an open set and F : ^ an analytic function. Take 
Xq ^ Q such that /3 := ||F'(a:;o)^-F(2;o) || > and F'{xq) is surjective. Suppose that 



7 := sup 

n>l 



F'(xo)1'FW(xo) 



ni 



l/(n-l) 



< +00, B{xo, 1/7) C Q. 



(41) 



Consider the auxiliary function /i^^^ : [0, 1/7)—)- M, 

:=/3-2t + t/(l -7t). 

// 

a := /37 < 3 - 2v^, 

i/ien hp^i^{t) has a smallest zero t, = (l+a-v/(l + a)2-8a)/(47), t/ie sequences for solving 
hp,K,{t) = and F{x) = 0, with starting point to = and xq, respectively, 

tk+i = tk - h'f^fi{tky^hf3^^{tk), Xk+i = Xk- F'{xk)^F{xk), A; = 0,1,..., 

are well defined, {tk} is strictly increasing, is contained in [0,t*), and converges Q-linearly 
to U, {xk} is contained in _B(xo,t*) and converges R-linearly to a point x* G B[xo,U] such 
that F'{x^yF{x^) = 0, 



tk+1 — tk 



fc = o,i,... , 



\F'{xo)^F{xk)\\ < 



{tk — tk-lY 

tk+l ~ tk 
tk — tk-1 



Xk Xk—i\\ , k — 1,2,..., 
F'{xoyF{xk-i)l A; = 1,2,.... 



//, additionally, a := /37 < 3 — 2\/2, then the sequences {tk} and {xk} converge Q- 
quadratically and R-quadratically to and x*, respectively. 



Proof. The proof follows the same pattern as the proof of Theorem [161 



□ 



4 Final remarks 

We presented a new semi-local convergence analysis of the Gauss- Newton method for solving 
([T]), where F satisfies ([3]), under a majorant condition. It would also be interesting to present 
a local convergence analysis of the Gauss-Newton method, under a majorant condition, for 
the problem under consideration. As a consequence, we would get convergence results for 
analytical functions under an 7-condition. This local analysis will be performed in the future. 
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